Facial recognition display control method and apparatus

ABSTRACT

A screen display control method is provided for controlling screen display of an electronic device more efficiently. A screen display control method includes determining a resolution of a camera, recognizing a face of a user using the camera operating at the determined resolution, and controlling screen display of the electronic device based on the face.

CROSS-REFERENCE TO RELATED APPLICATION(S)

This application claims the benefit under 35 U.S.C. § 119(a) of a Koreanpatent application filed on Mar. 14, 2013 in the Korean IntellectualProperty Office and assigned Serial No. 10-2013-0027592, and of a Koreanpatent application filed on Mar. 5, 2014 in the Korean IntellectualProperty Office and assigned Serial No. 10-2014-0025929, the entiredisclosure of each of which is hereby incorporated by reference.

TECHNICAL FIELD

The present disclosure relates to a method for controlling screendisplay of an electronic device efficiently.

BACKGROUND

With advances in information, communication, and semiconductortechnologies, the popularity and use of electronic devices continues toincrease. Particularly, a mobile terminal is integrating variousfeatures of other electronic devices beyond the voice telephony andmessaging functions. For example, the mobile terminal may now includevarious advanced functions including a TV playback function (e.g. amobile broadcasting playback function such as Digital MultimediaBroadcasting (DMB) and Digital Video Broadcasting (DVB)), a musicplayback function (e.g. MPEG Audio Layer-3 (MP3) playback function), apicture shooting function, a data communication function, an InternetAccess function, a short range wireless communication function, etc.

Also, the mobile terminal is equipped with various sensors as well as acamera to extend its functionality. For example, a camera-equippedmobile terminal is capable of recognizing a face from an image taken bythe camera and tracking the gaze of the user. However, the electronicdevice of the related art lacks functionality in view of powerutilization efficiency. Accordingly, there is a need for an improvedapparatus and method for controlling the display of an electronic devicein view of power demands.

The above information is presented as background information only toassist with an understanding of the present disclosure. No determinationhas been made, and no assertion is made, as to whether any of the abovemight be applicable as prior art with regard to the present disclosure.

SUMMARY

Aspects of the present disclosure are to address at least theabove-mentioned problems and/or disadvantages and to provide at leastthe advantages described below. Accordingly, an aspect of the presentdisclosure is to provide a display control method and apparatus of anelectronic device that is capable of controlling the screen display ofan electronic device depending on the power state of the electronicdevice or the presence/absence of the user regardless of the powerstate, thereby improving power utilization efficiency of the electronicdevice.

In accordance with an aspect of the present disclosure, a screen displaycontrol method of an electronic device is provided. The screen displaycontrol method includes determining a resolution of a camera,recognizing a face of a user using the camera operating at thedetermined resolution, and controlling screen display of the electronicdevice based on the face.

In accordance with another aspect of the present disclosure, anelectronic device is provided. The electronic device includes a controlunit configured to determine a resolution, a camera configured tooperate at the resolution determined by the control unit to recognize aface of a user, and a display unit configured to display a screen undera control of the control unit based on the recognized face.

Other aspects, advantages, and salient features of the disclosure willbecome apparent to those skilled in the art from the following detaileddescription, which, taken in conjunction with the annexed drawings,discloses various embodiments of the present disclosure.

BRIEF DESCRIPTION OF THE DRAWINGS

The above and other aspects, features, and advantages of certainembodiments of the present disclosure will be more apparent from thefollowing description taken in conjunction with the accompanyingdrawings, in which:

FIG. 1 is a flowchart illustrating a screen display control methodaccording to an embodiment of the present disclosure;

FIG. 2 is a flowchart illustrating a screen display control procedure ofan electronic device equipped with a high resolution camera according toan embodiment of the present disclosure;

FIGS. 3 and 4 are flowcharts illustrating screen display controlprocedures using a low resolution camera according to an embodiment ofthe present disclosure;

FIGS. 5A, 5B, 5C, 5D, and 5E are diagrams illustrating screen displaysfor explaining a screen display control method according to anembodiment of the present disclosure;

FIGS. 6A and 6B are diagrams illustrating a situation of controllingscreen display according to an embodiment of the present disclosure; and

FIG. 7 is a block diagram illustrating a configuration of an electronicdevice according to an embodiment of the present disclosure.

The same reference numerals are used to represent the same elementsthroughout the drawings.

DETAILED DESCRIPTION

The following description with reference to the accompanying drawings isprovided to assist in a comprehensive understanding of variousembodiments of the present disclosure as defined by the claims and theirequivalents. It includes various specific details to assist in thatunderstanding but these are to be regarded as merely exemplary.Accordingly, those of ordinary skill in the art will recognize thatvarious changes and modifications of the various embodiments describedherein can be made without departing from the scope and spirit of thepresent disclosure. In addition, descriptions of well-known functionsand constructions may be omitted for clarity and conciseness.

According to an embodiment of the present disclosure, the electronicdevice may be a device equipped with a communication function. Examplesof the electronic device include a smartphone, a tablet PersonalComputer (PC), a mobile phone, a video phone, an e-book reader, adesktop PC, a laptop PC, a netbook computer, a Personal DigitalAssistant (PDA), a Portable Multimedia Player (PMP), MP3 player, amobile medical appliance, a camera, and a wearable device (e.g. aHead-Mounted-Device (HMD) such as electronic glasses, electronicclothing, an electronic bracelet, an electronic necklace, an electronicappcessory, an electronic tattoo, a smart watch, etc.

According to an embodiment of the present disclosure, the electronicdevice may be a smart home appliance equipped with a communicationfunction. Examples of the smart home appliance include a television, aDigital Video Disk (DVD) player, an audio device, a refrigerator, anair-conditioner, a vacuum cleaner, an oven, a microwave oven, a laundrymachine, an air cleaner, a set-top box, a TV box (e.g. SamsungHomeSync™, Apple TV™, and Google TV™), a game console, an electronicdictionary, an electronic key, a camcorder, an electronic frame, etc.

According to an embodiment of the present disclosure, examples of theelectronic device include various medical appliances (e.g. MagneticResonance Angiography (MRA), Magnetic Resonance Imaging (MRI), ComputedTomography (CT) camera, and ultrasonic device), a navigation device, aGlobal Positioning System (GPS) receiver, an Event Data Recorder (EDR),a Flight Data Recorder (FDR), a car infotainment device, a marineelectronic device (e.g. marine navigation device and gyro compass), anavionics device, a security device, industrial and home robots, etc.

According to an embodiment of the present disclosure, examples of theelectronic device include furniture or part of building/constructionwhich is equipped with a communication function, an electronic board, anelectronic signature receiving device, a projector, a metering device(e.g. water, electricity, gas, and electric wave metering devices). Theelectronic device according to an embodiment of the present disclosuremay be any combination of at least two of the above-enumerated devices.It is obvious to those skilled in the art that the electronic device ofthe present disclosure is not limited to the above-enumerated devices.

FIG. 1 is a flowchart illustrating a screen display control methodaccording to an embodiment of the present disclosure. The screen displaycontrol method according to an embodiment of the present disclosure isexecuted by an electronic device.

Referring to FIG. 1, at operation 110, the electronic device maydetermine the resolution of a camera. The resolution denotes a degreeindicating how many pixels or dots are used for presenting the image.For example, the electronic device may accomplish the ‘high resolution’by activating all of the pixels of the image sensor of the camera andthe ‘low resolution’ by activating a part of the pixels of the imagesensor. The electronic device may determine the resolution of the cameraregardless of or depending on the power state. In an embodiment, theelectronic device may set the resolution of the camera to high or lowregardless of the power state.

In another embodiment, the electronic device may set the resolution ofthe camera to high or low depending on its power state. For example, ifthe power of the electronic device is equal to or greater than athreshold, the electronic device determines the current state as thehigh power state and sets the camera resolution to high. Otherwise ifthe power of the electronic device is lower than the threshold, theelectronic device determines the current state as the low power stateand sets the camera resolution to low. Here, the threshold may beexpressed as the percentage of the residual power of the electronicdevice and may be variously set, such as to 30%, 40%, 50%, or 60% of thebattery capacity. The threshold may be set to a default value by theelectronic device or to a certain value by the user. For example, if thethreshold value is set to 30% of the battery capacity, the electronicdevice determines the residual power state of 30% as a high power stateand the residual power state of 29% as a low power state.

If the electronic device is connected to an external device, itdetermines its state as the high power state and sets the cameraresolution to high. If the electronic device is not connected to theexternal device and if its power state is low, it sets the cameraresolution to low. The external device is a device supplying power tothe electronic device, for example through a Universal Serial Bus (USB)or a node such as a PIN CONNECTOR. In this case, since the externaldevice supplies power to the electronic device, the electronic devicedetermines its power state as the high power state regardless of theresidual power. Otherwise, if the external device with low residualpower is not connected to the external device, it determines that itsstate is the low power state.

At operation 120, the electronic device activates the camera with thedetermined resolution to recognize the face of the user. As an example,face recognition recognizes the face or gaze of the user to determinewhether it is the front of the user's face or a side of the user's faceand recognizes the distorted degree of the face.

According to various embodiments, the electronic device may accomplishthe ‘high resolution’ by activating all of the pixels of the imagesensor of the camera and the ‘low resolution’ by activating a part ofthe pixels of the image sensor. Assuming that the total number of pixelsof the image sensor is 10 million pixels, the electronic device sets athreshold to 5 million. The electronic device may determine the numberof pixels equal to or greater than the threshold as ‘high resolution’and the number of pixels less than the threshold as ‘low resolution.’

At operation 130, the electronic device may control the screen displayof the electronic device based on the recognized face. According tovarious embodiments, the electronic device operates the camera with highresolution to acquire the gaze position of the user and controls thescreen display based on the gaze position. The electronic device alsomay operate the camera with low resolution to determine thepresence/absence of the user and if it is determined that a user ispresent, change configuration values of the camera to recognize the faceposition of the user and control the screen display based on the faceposition.

The electronic device may control at least one action associated withthe screen displayed on the display unit such as scroll, playback,pause, zoom-in, zoom-out, stay, and response based on the recognizedface. The term scroll denotes a screen control of scrolling up, down,left, and right. The term playback denotes a screen control of playingmusic or motion picture on the screen. The term pause denotes a screencontrol of pausing the music and motion picture. The term zoom-indenotes a screen control of zooming in the screen, and the term zoom-outdenotes a screen control of zooming out the screen. The term staydenotes a screen control of maintaining the screen as turned on, and theterm response denotes controlling the screen according to the userresponse to an incoming call or message. As an example, the userresponse may be one of receiving and rejecting the incoming call,determining whether to view a message or not, and the like.

According to an embodiment, when the electronic device is connected toan external device, it is possible to use the power without restrictionand thus to operate the camera at the maximum frame rate to improve theuser recognition probability. The electronic device may further supporta monitoring function, a fire protection function, and an emergencysituation recoding function.

FIG. 2 is a flowchart illustrating a screen display control procedure ofan electronic device equipped with a high resolution camera according toan embodiment of the present disclosure.

Referring to FIG. 2, at operation 210 a, the electronic device acquiresthe gaze position of the user using the high resolution camera. Thecamera may be arranged at the front or rear side of the electronicdevice to capture a surrounding scene, an object, or a person. Thecaptured image information is sent to the control unit of the electronicdevice and stored in the electronic device. The camera may be aninfrared camera or other type of camera. The high resolution camera is acamera capable of operating in a high resolution mode capable of beingconfigured with high resolution color values to recognize a user's faceand track the gaze position of the user simultaneously or sequentially.For example, the electronic device is capable of recognizing a user'sface by combining the orientations of the pupils of the user's face todetermine the gaze position.

At operation 220 a, the electronic device acquires coordinateinformation by mapping the gaze position to the display unit of theelectronic device. The electronic device may also acquire the coordinateinformation of the gaze position on the display unit which correspondsto the gaze position. According to various embodiments, the electronicdevice may control the screen display of the electronic device basedonly on the acquired coordinate information. However, since the gazeposition of the user varies over time, the electronic device determinesa movement direction or a movement speed of the coordinate informationto control the screen display more accurately.

At operation 230 a, the electronic device determines directioninformation or speed information based on the change of the coordinateinformation over time. The electronic device is capable of determiningthe direction information or the speed information based on theinitially acquired gaze position and the changed gaze position if thegaze position changes over time since the initial recognition of thegaze. According to various embodiments, the electronic device maycompensate the direction information and the speed information usingsensor data acquired by means of a sensor.

At operation 240 a, the electronic device controls the screen displaybased on the direction or speed information. For example, if the screendisplay control is “scroll”, the direction or speed information mayaffect the scroll direction and/or scroll speed.

In another embodiment, the electronic device is capable of acquiring thesensor data using a sensor at operation 220 b. The sensor may include atleast one of a Motion Sensor, a Gyro Sensor, an Acceleration Sensor, aHumidity Sensor, a Proximity Sensor, an Infrared Sensor, an IlluminanceSensor, an Earth Magnetic Field Sensor, an Inertial Sensor, a TiltSensor, and a Blow Sensor (e.g., an exhale sensor). The sensor mayacquire the sensor data including at least one or any combination of anangle change, a direction change, a position change, a posture change, astrength change, and a speed change. The sensor detects a motion eventoccurred by the user so as to acquire the sensor data from at least oneor any combination of a pinch zoom, a hand blade touch, and an airmotion. For reference, the blow sensor is capable of detecting a blow ofthe user, and the sound detection sensor such as a speaker is capable ofdetecting the blow sound.

At operation 230 b, the electronic device may assign the priority of theimage information and the sensor data. Here, the image information isthe information acquired by the camera such as a gaze position. Forexample, if the camera is a high resolution camera, the electronicdevice may set the priority of the image information to a value higherthan that of the sensor data. If the camera is a low resolution camera,the electronic device may set the priority of the image information to avalue lower than that of the sensor data.

At operation 240 b, the electronic device controls the screen displayusing the information sorted in a descending order of priority. Forexample, if the priority of the image information is high, theelectronic device may control the screen display based on the imageinformation. Otherwise, if the priority of the sensor data is high, theelectronic device may control the screen display based on the sensordata. For example, if the priority of the sensor data is higher thanthat of the image information, the electronic device may control thescreen display based on the sensor data. The image information may bereflected after the sensor data-based screen display control or used forcompensating the sensor data.

FIGS. 3 and 4 are flowcharts illustrating screen display controlprocedures using a low resolution camera according to an embodiment ofthe present disclosure.

Referring to the embodiment of FIG. 3, the electronic device determineswhether the user is present or absent using the low resolution camera atoperation 310 a. The low resolution camera is a camera operating in alow resolution mode implemented by setting configuration values to lowresolution black and white values. The electronic device consumes lowpower in the low resolution mode as compared to the high resolutionmode.

If it is determined that the user is present, the electronic devicechanges the configuration value of the camera at operation 320 a. Again,the camera is operating as a low resolution camera. The configurationvalue may be any of camera speed, i.e., Frames Per Second (FPS), andInternational Organization for Standardization (IOS). The electronicdevice also may change at least one of a diaphragm of camera, a shutterspeed, an Exposure Value (EV), a White Balance (WB), and an exposuretime. The electronic device is capable of increasing the recognitionrate of the camera by changing the configuration values of the camera.

At operation 330 a, the electronic device recognizes the face positionof the user using the camera with the changed configuration values. Theface position may be the position where the user's face is located.

At operation 340 a, the electronic device controls the screen display ofthe electronic device based on the face position.

In another embodiment, the electronic device acquires the sensor datausing at least one sensor at operation 340 b. Since the sensor and thesensor data are identical with those described with reference to FIG. 2,a description thereof is omitted herein. For reference, the electronicdevice may determine whether the illuminance is sufficient to recognizea face by means of the camera based on the sensor data of theilluminance sensor. The electronic device may determine whether toperform image processing when the camera is running in the callprocessing based on the sensor data of the proximity sensor. Theelectronic device may also determine whether it is possible to acquireinformation consistently from the camera due to the upside-down postureor significant trembling of the electronic device based on the sensordata of the motion sensor.

At operation 350 b, the electronic device acquires the coordinateinformation by mapping the sensor data and face position to the displayunit of the electronic device. The electronic device may acquire thecoordinate information on the position of the display unit whichcorresponds to the gaze position. According to various embodiments, theelectronic device may control the screen display of the electronicdevice based only on the acquired coordinate information.

At operation 360 b, the electronic device determines the direction andspeed based on a change of the coordinates over time. If the gazeposition changes over time since the gaze position has been detected,the electronic device may determine the movement direction or speed ofthe gaze position using the initial gaze position and the changed gazeposition. According to various embodiments, the electronic device maycompensate the direction information and/or the speed information usingthe sensor data acquired by the sensor.

At operation 370 b, the electronic device may control the screen displaybased on the direction information and/or the speed information.

Referring to FIG. 4, the electronic device determines whether the useris present or absent using the low resolution camera at operation 410 a.

If it is determined that the user is present, the electronic deviceacquires the sensor data using the sensor at operation 420 a.

At operation 430 a, the electronic device controls the screen displaybased on the sensor data. Accordingly, the electronic device can controlthe screen display based on only the sensor data. The electronic devicemay control the screen display using the information on the faceposition recognized using the low resolution camera along with thesensor data.

In an embodiment, the electronic device determines whether it ispossible to acquire an image from the camera based on the sensor data atoperation 430 b. The electronic device analyzes the sensor data acquiredfrom the illuminance sensor to determine whether the ambient illuminanceis equal to or greater than a threshold and thus whether it is possibleto acquire an image using the camera. The electronic device may analyzethe sensor data from the proximity sensor to determine whether thedistance from the face position of the user is within a thresholddistance and thus whether it is possible to acquire an image using thecamera.

If it is determined that the image acquisition is possible, theelectronic device changes the configuration value of the camera based onthe sensor data at operation 440 b. The configuration value may be thecamera speed, i.e. FPS, or camera sensitivity.

At operation 450 b, the electronic device may recognize the faceposition of the user using the camera configured with the changedconfiguration value.

At operation 460 b, the electronic device controls the screen displaybased on the face position of the user.

Otherwise if it is determined that the image acquisition is impossible,the electronic device stops the operation of the camera and controls thescreen display based on the sensor data at operation 470 b. That is, theelectronic device may control the screen display based on only thesensor data.

FIGS. 5A to 5E are diagrams illustrating screen displays for explaininga screen display control method according to an embodiment of thepresent disclosure.

Referring to FIGS. 5A to 5E, the electronic device 500 may operate thecamera in one of the low and high resolution modes depending on orindependently of the power state.

FIG. 5A shows a situation of controlling the screen display in responseto a scroll (smart scroll) command. If the smart scroll command isexecuted, the electronic device 500 invokes the application supportingthe scroll. The electronic device may scroll the content displayed onthe display unit to the left, right, up, or down in correspondence tothe face recognized by the camera. For example, if the first gazeposition 510 changes for the second gaze position 520 on the displayunit of the electronic device 500, the electronic device 500 scrolls thescreen bottom to top on the display unit such that the lower part of thecontent appears on the screen. The electronic device 500 is capable ofdetecting a drop of the face or a raise of the face based on at leastone of the gaze position, face position, and face angle to determine thescroll direction and/or speed. The electronic device 500 may scroll thescreen according to the determined scroll direction and/or speed.

According to various embodiments of the present disclosure, theelectronic device 500 may compensate the scroll direction and/or scrollspeed based on the sensor data acquired from the sensor. For example,the electronic device 500 determines whether the user's gaze positionacquired through the camera corresponds to the display unit using thetilt sensor and, if so, scrolls the screen using only the sensor data.The gyro sensor senses the leaning degree of the electronic device 500and controls the scroll direction and/or scroll speed according to theleaning degree. For example, if the electronic device 500 leans upwardtwice in quick motion, the screen may be scrolled upward quickly. Atthis time, if the electronic device 500 leans download once, the upwardscrolling stops. Also, if the electronic device leans downward twice,the upward scroll stops and the downward scroll starts. The user maymanually configure at least one of the direction, angle, and speed. Ifthe acquired sensor data corresponds to at least one of the direction,angle, and speed information, the electronic device 500 scrolls thescreen according to the at least one of the direction, angle, and speedinformation.

According to various embodiments, if it is impossible to recognize theface of the user using the camera, the electronic device 500 may acquirethe sensor data from the sensor. The electronic device 500 determineswhether it is not manipulated by the user based on the sensor data. Forexample, if it is determined that the electronic device 500 maintains aposture lying flat horizontally as if it is placed on the desk withoutany input, the electronic device 500 determines that it is not in use.If it is determined that the electronic device is not in use, theelectronic device may decrease the frame rate of the camera. If no inputis detected in a threshold time, the electronic device 500 stopsoperating the camera. That is, the electronic device determines whetherit is in use based on the sensor data and adjusts the frame rate of thecamera to turn off the camera if it is determined that the electronicdevice 500 is not in use.

FIG. 5B shows a situation of controlling the screen display in responseto a playback (smart play) command. For example, the electronic device530 may display music content. If the gaze position is on the displayunit displaying the music content, the electronic device 540 may playthe music content.

FIG. 5C shows a situation of controlling the screen display in responseto a pause (smart pause) command. If the gaze position of the user is onthe display unit, the electronic device plays the music content as shownin the screen display 550. In the state of playing the music content, ifthe gaze position of the user moves out of the display unit, theelectronic device pauses the playback of the music content as shown inthe screen display 560.

FIG. 5D shows a situation of controlling the screen display in responseto a zoom-in (smart zoom-in) or a zoom-out (smart zoom-out) command. Theelectronic device 570 may zoom in or out the content according to thedistance between the gaze position of the user and the display unit. Theelectronic device 570 may display the content in a threshold distance(d₀) between the gaze position and the display unit. If the distancebetween the gaze position and the display unit (d₁) is equal to orgreater than the threshold distance (d₀) (d₀≤d₁), the electronic devicemay zoom in the content as shown in the drawing 570 a. If the distancebetween the gaze position and the display unit (d₂) is less than thethreshold distance (d₀) (d₀>d₂), the electronic device may zoom out thecontent as shown in the drawing 570 b.

FIG. 5E shows a situation of controlling the screen display in responseto a response command. If the gaze position is on the display unit, theelectronic device may accept the incoming call as shown in the drawing580 a. That is, if the gaze position moves from left to right as shownin the drawing 580 a, the electronic device receives the incoming call(response: yes). Otherwise if the gaze position moves from right to leftas shown in the drawing 580 b, the electronic device may reject theincoming call (response: no).

According to various embodiments, the electronic device may control thescreen display in response to the stay (smart stay) command. The staycommand is to instruct to maintain the screen displayed on the displayunit of the electronic device as it is turned on. In the case of thestay command, the electronic device requires a wide angle of view andmay use the front camera operating in YUV mode to reduce erroneous facerecognition probability.

FIGS. 6A and 6B are diagram illustrating a situation of controllingscreen display according to an embodiment of the present disclosure.

Referring to FIG. 6A, the electronic device 600 may execute a bookapplication such that the display unit displays a page as shown in thedrawing 610. If the motion sensor detects that the electronic devicestanding erect falls flat on its back as denoted by reference number 600a, the electronic device turns the current page to the next page asshown in the drawing 620. If it is detected that the gaze position ofthe user is placed on the right edge of the current page, the electronicdevice 600 may turn the current page to the next page on the displayunit.

Referring to FIG. 6B, the electronic device 600 may control the screendisplay of an external device 640 connected thereto. The externalelectronic device 640 may be any of all the types of electronic devicesconnectable from outside of the electronic device 600 such as atelevision, a computer monitor, a set-top box, a navigator, a speaker,and the like. At this time, the electronic device 600 may be connectedwith the external electronic device 640 via an external device 630. Inthe case that the electronic device 600 is connected with the externalelectronic device 640, it is possible to control the screen display ofthe external electronic device 640 based on the user's face recognizedby means of the camera. For example, if the recognized user's faceposition is on the external electronic device 640, the electronic device600 may continue providing the screen data to the external electronicdevice 640. If the recognized user's face position is output of thedisplay unit and the external electronic device 640, the electronicdevice 600 may stop playing the content at the external electronicdevice 640. If the face position is out of the display unit and theexternal electronic device 640 over a threshold time, the electronicdevice 600 may turn off the external electronic device 640 or controlthe external electronic device 640 to enter the power saving mode.

According to various embodiments, if the external electronic device 640is a speaker, the electronic device 600 may output the voice of acounterpart user of a call through the speaker in the telephony mode. Ifthe external electronic device 640 is a navigation device or an inbuiltcar device, the electronic device 600 may detect an emergency situationbased on the face and motion of the user. For example, the emergencysituation may be one of dozing, diverting attention away from the road,or finding an obstacle on the road. If it is determined that the user isdozing, the electronic device 600 may output an alarm sound orannouncement or notify of the situation on the road.

FIG. 7 is a block diagram illustrating a configuration of an electronicdevice according to an embodiment of the present disclosure.

Referring to FIG. 7, the electronic device 700 includes a control unit710, a camera unit 720, a display unit 730, an input unit 740, a powersupply unit 750, a sensor unit 760, and a communication unit 770.

The control unit 710 determines the resolution of the camera unit 720.According to various embodiments of the present disclosure, the controlunit 710 may set the resolution of the camera unit 720 to higher or lowregardless of the power state of the electronic device 700. The controlunit 710 may determine whether the electronic device is in the higherpower state or low power state depending on whether the power of theelectronic device 700 is equal to or greater than a threshold and/orwhether the electronic device is connected to an external device. Forexample, if the power of the electronic device 700 is equal to orgreater than the threshold or if the electronic device 700 is connectedto the external device, the control unit 710 determines this as the highpower state and sets the resolution of the camera unit 720 to high. Theexternal device supplies power to the electronic device 700. If thepower of the electronic device 700 is lower than the threshold and ifthe external device is not connected to the electronic device 700, thecontrol unit 710 determines this as the low power state and sets theresolution of the camera to low. The electronic device is supplied powerthrough the power supply unit 750. The power supply unit may beconnected to the external device.

The control unit 710 controls overall operations of the electronicdevice 700 and signal flows among the internal components of theelectronic device 700, processes data, and controls power supply to thecomponents. The control unit 710 includes a Central Processing Unit(CPU) and a Graphic Processing Unit (GPU). As well known in the art, theCPU is a core control unit of the computer system which performs dataoperation and comparison and command interpretation and execution. TheGPU is the graphic control unit of performing graphic data operation andcomparison and command interpretation and execution. Each of the CPU andGPU may be integrated into a package of a single integrated circuitincluding two or more independent cores (e.g. quad-core). The CPU andGPU also may be integrated into a chip in the form of a System on Chip(SoC). The CPU and GPU also may be implemented in the form ofmulti-layered package. The packaged CPU and GPU may be referred to asApplication Processor (AP).

The camera 710 may operate at the determined resolution to recognize theface of the user. According to various embodiments of the presentdisclosure, the camera unit 720 operates at a high resolution to acquirethe gaze position of the user. The camera unit 720 may operate at a lowresolution to recognize the face position of the user.

The display unit 730 may display a screen controlled based on the face.The display unit 730 displays an image on the screen under the controlof the control unit 710. That is, the control unit 710 processes thedata to generate an image fit for the screen (e.g. decoding the data) tothe buffer, and the display unit 730 converts the buffered image toanalog signals such that the image is presented on the screen. Thedisplay unit 730 may be implemented with any of a Liquid Crystal Display(LCD), an Organic Light Emitting Diodes (OLED), an Active Matrix OLED(AMOLED), and a flexible display. According to various embodiments ofthe present disclosure, the display unit 730 may be integrated with theinput unit 740.

The input unit 740 includes a plurality of keys for receivingalphanumeric information and configuring various functions. Examples ofthe keys include a menu key, a screen on/off key, a power on/off key, avolume control key, etc. The input unit 740 generates user setting andelectronic device function control key signals to the control unit 710.Examples of the key signal include a power on/off signal, a volumecontrol signal, a screen on/off signal, a shutter signal, etc. Thecontrol unit 710 controls the above components in response to the keysignal. The keys constituting the input unit 740 are referred to as hardkeys and the keys presented on the display unit 730 are referred to assoft keys.

According to various embodiments, the control unit 710 acquires thecoordinate information by mapping the gaze position to the display unit730 and control the screen display based on the acquired coordinateinformation. The control unit 710 determines the direction and speedinformation based on the change in coordinate information and controlsthe screen display based on the direction and speed information. Thecontrol unit 710 also determines whether the user is present or absentusing the camera unit 720 operating in the low resolution mode and, ifthe user is present, changes the configuration value of the camera unit720 to recognize the face position of the user. The configuration valuemay be the camera speed, i.e. FPS, or camera sensitivity.

The sensor unit 760 may include at least one of a Motion Sensor, a GyroSensor, an Acceleration Sensor, a Humidity Sensor, a Proximity Sensor,an Infrared Sensor, an Illuminance Sensor, an Earth Magnetic FieldSensor, an Inertial Sensor, a Tilt Sensor, and a Blow Sensor. The sensormay acquire the sensor data including at least one or any combination ofan angle change, a direction change, a position change, a posturechange, a strength change, and a speed change. The sensor detects amotion event occurred by the user so as to acquire the sensor data fromat least one or any combination of a pinch zoom, a hand blade touch, andan air motion.

According to various embodiments, the control unit 710 receives thesensor data from the sensor unit 760 to acquire the coordinateinformation based on the sensor data and the mapping of the faceposition to the display unit of the electronic device. The control unit710 determines the direction and speed information based on the changeof coordinate information over time. That is, the control unit 710acquires the coordinate information using the face position and thesensor data and determines the movement direction and speed accurately.

According to various embodiments, if it is determined that the user ispresent, the control unit 710 activates at least one sensor of thesensor unit 760 and controls the screen display based on the sensor dataprovided by the sensor unit 760. The control unit 710 determines whetherit is possible to acquire image from the camera based on the sensor dataand, if so, changes the configuration values of the camera based on thesensor data to recognize the face position of the user. If it isimpossible to acquire image from the camera, the control unit 710 stopsoperating the camera unit 720 and controls the screen display based onthe sensor data.

According to various embodiment, the control unit 710 may assign thepriorities of the image information acquired using the camera unit 720and the sensor data. The control unit 710 may control the screen displaybased on the information sorted in the descending order. For example, ifthe camera unit 720 operates in the high resolution mode, the controlunit 710 may assign a high priority to the image information as comparedto the sensor data. Otherwise if the camera unit 720 operates in the lowresolution mode, the control unit 710 may assign a low priority to theimage information as compared to the sensor data.

In this way, the control unit 710 is capable of controlling the screendisplay of the display unit 730 in association with at least one of thescroll, playback, pause, zoom-in, zoom-out, stay, and response command.

The electronic device 700 may also function as at least one of an outputunit (not shown), a communication unit 770, and a memory (not shown).

The output unit may include a microphone and a speaker. The output unitmay input and output an audio signal (e.g. voice data) concerning voicerecognition, voice recording, digital recording, and telephony. Theoutput unit converts the audio signal from the control unit 710 to ananalog signal (D/A conversion) and amplifies and outputs the signalthrough the speaker. The output unit convers the audio signal (firstsound wave) received through the microphone to the digital signal (A/Dconversion) and transfers the digital signal to the control unit 710.The speaker converts the audio signal (sound data) to sound wave. Themicrophone converts the sound wave generated by a person or other soundsource to an audio signal.

The communication unit 770 is responsible for voice, video, and/or datacommunication with another electronic device through a network under thecontrol of the control unit 710. The communication unit 770 includes aRadio Frequency (RF) transmitter for frequency up-converting andamplifying of the transmission signal and an RF receiver for low noiseamplifying and frequency down-converging of the received signal. Thecommunication unit 770 may include a mobile communication module (e.g.3^(rd) Generation (3G) mobile communication module, 3.5G mobilecommunication module, and 4G mobile communication module), a digitalbroadcast module (e.g. DMB module, and a short range communicationmodule (e.g. Wi-Fi module, Bluetooth module, and Near FieldCommunication (NFC) module).

The memory may store software applications concerning the operation ofthe electronic device 700 and various data received or generated as aresult of the operation of the electronic device 700. The memory may bea certain type of digital storage readable and recordable. The memorymay store at least one program for executing the above functions. Thememory may include a secondary memory unit of the control unit 710 suchas disk, Random Access Memory (RAM), Read Only Memory (ROM), and flashmemory. The memory may store the information (e.g. contact) generated bythe electronic device 700 and the data (e.g. message and motion picturefile) received from the outside through the communication unit 770. Thememory may store the information on the image (e.g. keypad, motionpicture, and message) such as size information and display regioninformation.

As described above, the display control method and apparatus of thepresent disclosure is advantageous in that the screen display of theelectronic device is controlled depending on the presence/absence of theuser regardless of the power state of the electronic device so as toimprove power utilization efficiency of the electronic device.

Also, the display control method and apparatus of the present disclosureis advantageous in that the electronic device executes the functionsaffecting power consumption selectively depending on its power state soas to improve power utilization efficiency.

While the present disclosure has been shown and described with referenceto various embodiments thereof, it will be understood by those skilledin the art that various changes in form and details may be made thereinwithout departing from the spirit and scope of the present disclosure asdefined by the appended claims and their equivalents.

What is claimed is:
 1. A screen display control method of an electronicdevice, the method comprising: determining a resolution operation modeof a camera, the camera being configured to operate in a high resolutionmode by activating all pixels of an image sensor of the camera and a lowresolution mode by activating a predetermined number of pixels less thanthe all pixels of the image sensor of the camera; performing a firstfacial recognition operation when it is determined that the camera isoperating in the high resolution mode; performing a second facialrecognition operation when it is determined that the camera is operatingin the low resolution mode; and controlling a screen display of theelectronic device based on the first facial recognition operation or thesecond facial recognition operation, wherein the first facialrecognition operation includes determining a gaze position of a user,wherein the second facial recognition operation includes: setting firstconfiguration values of the camera, determining whether the user ispresent, when the user is present, changing the first configurationvalues of the camera to a second configuration value, and determining aface position of the user with respect to the electronic device whileoperating the camera in the low resolution mode with the secondconfiguration value, and wherein the changing of the first configurationvalues of the camera to the second configuration value compriseschanging at least one of frames per second, a shutter speed, or anexposure value.
 2. The method of claim 1, further comprising:determining a power state of the electronic device; and selecting thehigh resolution mode or the low resolution mode based on the determinedpower state of the electronic device.
 3. The method of claim 2, whereinthe determining of the power state comprises: determining, when thepower state of the electronic device is equal to or greater than athreshold or the electronic device is connected to an external device,the power state as a high power state; and determining, when the powerstate of the electronic device is less than the threshold value and theelectronic device is not connected to the external device, the powerstate as a low power state.
 4. The method of claim 3, wherein the highresolution mode is selected when the electronic device is in the highpower state, and wherein the low resolution mode is selected when theelectronic device is in the low power state.
 5. The method of claim 1,when the screen display is controlled based on the gaze position, themethod further comprising: acquiring coordinate information by mappingthe gaze position to a display of the electronic device; and controllingthe screen display further based on the coordinate information.
 6. Themethod of claim 5, further comprising: determining at least one ofdirection information or speed information based on a change of thecoordinate information over time; and controlling the screen displayfurther based on the direction information or the speed information. 7.The method of claim 1, further comprising: acquiring sensor data usingat least one sensor; acquiring coordinate information through mapping tothe display of the electronic device based on the sensor data and theface position; determining at least one of direction information andspeed information based on a change of the coordinate information; andcontrolling the screen display further based on the at least one of thedirection information and the speed information.
 8. The method of claim1, wherein, in the high resolution mode, the method further comprising:acquiring sensor data from a sensor; assigning priorities to imageinformation provided by the camera and the sensor data; and controllingthe screen display further using the image information sorted in adescending order of the priorities.
 9. The method of claim 1, whereinthe controlling of the screen display comprises controlling at least oneof scroll, playback, pause, zoom-in, zoom-out, stay, and responsecontrol operations in association with the screen display.
 10. Anelectronic device comprising: a camera configured to operate in aplurality of resolution operation modes, the plurality of resolutionoperation modes including a high resolution mode based on an activationof all pixels of an image sensor of the camera and a low resolution modebased on an activation of a predetermined number of pixels less than theall pixels of the image sensor of the camera; a display configured todisplay a screen; and at least one processor configured to: determinethe resolution operation mode of the camera, perform a first facialrecognition operation when it is determined that the camera is operatingin the high resolution mode, perform a second facial recognitionoperation when it is determined that the camera is operating in the lowresolution mode, and control the display based on the first facialrecognition operation or the second facial recognition operation,wherein the first facial recognition operation includes determining agaze position of a user, wherein the second facial recognition operationincludes: setting first configuration values of the camera, determiningwhether the user is present, when the user is present, changing thefirst configuration values of the camera to a second configurationvalue, and determining a face position of the user with respect to theelectronic device while operating the camera in the low resolution modewith the second configuration value, and wherein the changing of thefirst configuration values of the camera to the second configurationvalue comprises changing at least one of frames per second, a shutterspeed or an exposure value.
 11. The electronic device of claim 10,wherein the at least one processor is further configured to: determine apower state of the electronic device as one of a high power state and alow power state based on whether power of the electronic device is equalto or greater than a threshold or whether the electronic device isconnected to an external device, and select the high resolution mode ofthe camera when the electronic device is in the high power state and thelow resolution mode when the electronic device is in the low powerstate.
 12. The electronic device of claim 10, wherein the at least oneprocessor is further configured to: operate the camera in the highresolution mode to acquire coordinate information by mapping the gazeposition to the display of the electronic device, and control the screenfurther based on the coordinate information.
 13. The electronic deviceof claim 10, further comprising: at least one sensor configured toprovide sensor data, wherein the at least one processor is furtherconfigured to: acquire coordinate information through mapping to thedisplay of the electronic device based on the sensor data and the faceposition, determine at least one of direction information or speedinformation based on change of the coordinate information, and controlthe screen further based on the at least one of the direction or speedinformation.
 14. The electronic device of claim 10, wherein, in the highresolution mode, the electronic device further comprising: at least onesensor configured to provide sensor data, wherein the at least oneprocessor is further configured to: assign priorities to imageinformation provided by the camera and the sensor data, and control thescreen further using the image information sorted in a descending orderof the priorities.
 15. The electronic device of claim 10, wherein the atleast one processor is further configured to control at least one ofscroll, playback, pause, zoom-in, zoom-out, stay, and response controloperations in association with the screen.